As is known in the art, many data storage systems periodically store a copy, or replica, of the data currently in such system at a remote storage system. Former replication approaches require remote storage systems to have some “intelligence” in order to prevent transmission errors corrupting the data copy at the remote storage system.
More particularly, many applications maintain persistent state information by storing data on disk. Often the data stored on disk is designed to allow an application to return to the same state after an unexpected restart. The ability of an application to reliably return to a previous state often depends on data being stored to disk in a specific order. In order to protect against data loss and business interruption due to disasters, application data is often replicated to a remote site. In many cases this replicated data is intended to allow an affected application to be restarted at the remote site and return to its pre-failure state. As a result, data must be copied to the remote site in the appropriate order. At present, there are three general methods to reliably maintain a remote copy suitable for application restart: synchronous, semi-synchronous and periodic replication. Conventionally, all three methods require the remote storage systems to have some degree of intelligence in order to ensure that the remote copy remains point-in-time consist with the source data when errors occur. This requirement can prohibit performing consistent replication between dissimilar storage systems either from different vendors or different models from the same vendor. At present, this requirement prohibits performing replication between dissimilar systems: either storage systems from different vendors or, in some cases, different storage system models from the same vendor.
Still more particularly, many applications maintain persistent state information by storing data on disk. Often the data stored on disk is designed to allow an application to return to the same state after a planned or unexpected restart. To ensure that they can return to the same state applications strictly control the order in which state information is written to disk. Typically, I/O requests (e.g., a request from a host computer to store data in a disk (i.e., a write I/O request), or a request from the host computer to obtain data from the disk (i.e., a read I/O request)) are not issued until I/O operations to store previous state information have completed. Such write requests are said to be dependent on the previous write requests. Applications rely on this explicit control of dependent write ordering to ensure that there will be no “holes” in the state information stored on disk.
In order to guarantee that this will be the case, disk storage systems must store write data in the order that it is received. In cases where write data is first stored in a cache memory prior to its being stored in the disk, the data storage system must make certain that the data for all successfully completed write I/Os is always written to disk.
Saving state information to disk protects against data loss from an application or server failure. It does not, however, prevent data loss in the event of a disaster that affects the entire data center. Examples of such disasters include fires, earthquakes, and other similar catastrophic events.
As is also known in the art, Disaster Recovery (DR) solutions are commonly used to prevent these kinds of events from destroying data for critical applications. Often, an important component of such solutions is to maintain a copy of critical application data, including state information, at a geographically remote location. Ideally the remote location is far enough away from the primary data center to ensure that a single disaster (within reason) will not be able to destroy both data centers. In the event of a disaster, the remote copy of data can be used to either reconstruct a new primary datacenter or restart the affected applications at the remote location itself. In either case the remote copy ensures that business will be able to continue despite the disaster.
The term remote “replication” is often used to refer to the act of maintaining a remote copy of disk data. Some advanced storage systems are capable of performing remote replication automatically in a manner transparent to applications. Such solutions relieve critical applications from the burden of managing the remote data copy and allow them to focus on performing their particular business function. More particularly, a remote copy of critical application data is subject to the same requirements as the original data. Specifically, dependent writes must be applied to it in the same order that they were to the original copy of data by the application. Otherwise, it may not be possible to use the remote copy to restart an application and restore it to its pre-failure state. Clearly there are two states that a remote copy can be in:                Consistent: the remote copy is identical to the source data and therefore can be used to restore an application to its current state.        Inconsistent: the remote copy does not match any previous state of the source data and therefore cannot be used to return an application to a valid state.There is, however, a third intermediate state:        Point-in-time Consistent: the remote copy is identical to the source data at some point in the past and can be used to return an application to a previously valid state.Essentially, a point-in-time consistent copy is an older version of the source data that has had an application's dependent writes up to some previous time applied to it in the correct order. A point-in-time consistent copy, therefore, can be used to return an application to the previously valid state it was in at the time represented by the point-in-time copy. It should be noted, however, that recovering from a point-in-time consistent copy does not return the application to the same state that it was in at the time of failure. The implication of this is that some data may be lost. While data loss is in general undesirable there are, however, applications that can tolerate it making a point-in-time replication scheme an acceptable solution.        
At present there are three general methods for maintaining remote copies of data suitable for application restart.                Synchronous remote replication: Here, each data modification (i.e. host write request) is simultaneously applied to both the local and remote copies. Modifications are strictly ordered and completed serially. As a result the local and remote copies of data are effectively always identical;        Semi-synchronous remote replication; Here, each data modification (i.e. host write request) is applied to the local copy of data and queued for later application to the remote copy. Modifications to the remote copy are strictly ordered and completed serially. As a result the remote copy is always a point-in-time consistent version of the source data. Some semi-synchronous replication schemes define a maximum queue depth which, when reached, causes replication to fall back to a completely synchronous behavior in order to limit the skew between the source and remote copies; and        Periodic remote replication; Here, consecutive data modifications (i.e. host write requests) are aggregated into a series of “change sets” based on some periodic interval such as elapsed time or the amount of data modified. Each change set is then atomically applied to the remote copy in the appropriate order. As a result the remote copy is always a point-in-time consistent version of the source data.        
These principal differences between these three methods are levels of consistency and performance. Synchronous replication can maintain consistent remote copies but the latency of keeping the remote copy up to date can have significant negative impacts on application performance. Semi-synchronous replication decouples the updating of the source and remote copies to yield better performance but as a result can only achieve point-in-time consistency. Finally periodic replication can achieve even greater levels of performance since it is able to apply all the modifications in a change set in parallel and merge multiple modifications to the same piece of data during a period into a single change to the remote copy. However, this batching of modifications prevents periodic replication from being able to achieve the same levels of consistency as either synchronous or semi-synchronous replication. In addition the unordered, parallel application of modifications in a change set renders the remote copy inconsistent while it is being updated. In general, choosing the appropriate method is highly situational and depends on a variety of factors including the tolerable amount of data loss.
Generally speaking, some replication schemes are special cases of others. For instance synchronous replication can be considered a special case of semi-synchronous replication with a maximum queue depth of 1. Likewise semi-synchronous replication can be considered a special case of periodic replication with a change set size of 1. These observations suggest the possibility of applying enhancements to one replication method to its special case derivative(s) as well.
While in transit to the remote location it is possible for data modifications to suffer errors. Unless prevented, these corrupted operations can ruin the integrity of the remote data copy rendering it inconsistent with the source data, and therefore unsuitable for disaster recovery. In order to ensure the integrity of the remote data copy, remote replication schemes often rely on the remote storage system having some amount of “intelligence” to prevent corrupted data modifications from being applied to the remote copy.
The serial nature of synchronous and semi-synchronous replication schemes results in only a single data modification to the remote copy being performed at a time. Therefore, for these schemes, maintaining the integrity of the remote copy is simply a matter of preventing a single corrupted modification from being applied. This implies that only a limited amount of intelligence is needed at the remote site to maintain integrity for these replication schemes.
To achieve greater levels of performance, periodic remote replication schemes often apply multiple modifications in a change set to the remote copy simultaneously. In addition, the modifications in a change set are often applied in no particular order. This behavior makes maintaining the integrity of the remote copy more problematic as recovering from an error not only involves preventing corrupted data from being applied to the remote copy but it also requires reversing all previous modifications from the change set that completed successfully. As result, conventional periodic replication schemes require the remote storage system to have significantly more intelligence to track the progress of and changes by change sets in order to successfully recover from errors.
Further, there is a need for remote storage systems to have some degree of intelligence often limits the interoperability of storage systems. At present, the lack of open standards prevents remote replication between storage systems from different vendors. Sometimes, similar issues prevent replication between different models of storage systems from the same vendor. As a result, remote replication schemes are often limited to specific configurations based on homogeneous storage systems.